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Abstract. Wc consider the dynamical behavior of Martin-L6f random points 
in dynamical systems over metric spaces with a computable dynamics and a 
, computable invariant measure. We use computable partitions to define a sort 

1 of effective symbolic model for the dynamics. Through this construction we 

' prove that such points have typical statistical behavior (the behavior which is 

' typical in the Birkhoff ergodic theorem) and are recurrent. We introduce and 

compare some notions of complexity for orbits in dynamical systems and prove: 
(i) that the complexity of the orbits of random points equals the Kolmogorov- 
' Sinai entropy of the system, (ii) that the supremum of the complexity of orbits 

' equals the topological entropy. 
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2 stefano galatolo, mathieu hoyrup, and cristobal rojas 

1. Introduction 

The randomness of a particular outcome is always relative to some statistical 
test. The notion of algorithmic randomness, defined by Martin-L6f in 1966, is an 
attempt to have an "absolute" notion of randomness. This absoluteness is actually 
relative to all "effective" statistical tests, and lies on the hypothesis that this class 
of tests is sufficiently wide. 

Martin-L6f 's original definition was given for infinite symbolic sequences. With 
this notion each single random sequence behaves as a generic sequence of the prob- 
ability space for each effective statistical test. In this way many probabilistic theo- 
rems having almost everywhere statements can be translated to statements which 
hold for each random sequence. As an example we cite the fact that in each infinite 
string of O's and I's which is random for the uniform measure, all the digits appear 
with the same limit frequency. This is a particular case, related to the strong law 
of large numbers (or Birkhoff ergodic theorem). A general statement of this kind 
was given by V'yugin (Birkhoff ergodic theorem for individual random sequences, 
see [V"y97] and lemma 3.2.1 below). 

Recently the notion of Martin-L6f randomness was generalized to computable 
metric spaces endowed with a measure ([Gac05, HR07]). Computable metric spaces 
are separable metric spaces where the distance can be in some sense effectively com- 
puted (see section 2.4 ). In those spaces, it is also possible to define "computable" 
functions, which are functions whose behavior is in some sense given by an algo- 
rithm, and "computable" measures (there is an algorithm to calculate the measure 
of nice sets). The space of infinite symbolic sequences, the real line or euclidean 
spaces, are examples of metric spaces which become computable in a very natural 
way. 

A particularly interesting class of general stationary stochastic processes is con- 
stituted by those generated by a measure-preserving map on a metric space, these 
are the objects studied by ergodic theory. In this paper we consider systems of the 
type (X, T, /i), where X is a computable metric space, fi a computable probability 
measure and T a computable endomorphism. The above considered symbolic shifts 
on spaces of infinite sequences which preserve a computable measure are systems 
of this kind. 

In the classical ergodic theory, a powerful technique (symbolic dynamics) allows 
to associate to a general system as above {X, T, n) a shift on a space of infinite 
strings having similar statistical properties. In section 3 we use the algorithmic 
features of computable metric spaces to define computable measurable partitions 
and construct effective symbolic models for the dynamics. In this models random 
points are associated to random infinite strings. This tool allows to generalize 
theorems which are proved in the symbolic setting to the more general setting of 
maps and metric spaces. For example the above cited V'yugin theorem becomes 
a Birkhoff theorem for random points. On this line, we also prove a Poincare 
recurrence theorem for random points. Those statements (see thm.3.2.1 and prop. 
3.2.1) can be summarized as 



Theorem. Let {X,fi) be a computable probability space. If x is ji-random, then it 
is recurrent with respect to every measure preserving endomorphism T on {X, fj,) . 
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Moreover, each ^-random point x is typical for every ergodic endomorphism T , i.e. 

n-l 

lim - ^ fiT^x) = / fdfi (1) 

n— >oo ri ^ — ^ / 

for every continuous bounded real-valued f . 

In the remaining part of the paper these tools are also used to prove relations 
between various definitions of orbit complexity and entropy of the systems. 

In [BrTi83], Brudno defined a notion of algorithmic complexity IC{x,T) for the 
orbits of a dynamical system on a compact space. It is a measure of the information 
rate which is necessary to describe the behavior of the orbit of x. In this point- 
wise definition the information is measured by the Kolmogorov information content. 
Later, White ([Whi93]) also introduced a slightly different version !C{x, T). Brudno 
then proved the following results, later improved by White: 

Theorem (Brudno, White). Let X he a compact topological space and T : X ^ X 
a continuous map. 

(1) For any ergodic probability measure fi the equality ]C{x,T) ~ K{x,T) = 
hfj_{T) holds for ji-almost all x £ X, 

(2) For all xeX, lC{x,T) < h{T). 

where hfj,{T) is the Kolmogorov-Sinai' entropy of {X^T) with respect to ^ and 
h{T) is the topological entropy of (X, T). This result seems miraculous as no 
computability assumption is required on the space or on the transformation T. 
Actually, this miracle lies in the compactness of the space, which makes it finite 
when observations are made with finite precision (open covers of the space can 
be reduced to finite open covers). Indeed, when the space is not compact, it is 
possible to construct systems for which the algorithmic complexity of orbits is 
correlated in no way to their dynamical complexity. In [GalOO], Brudno's definition 
was generalized to non-compact computable metric spaces. This definition coincides 
with Brudno's one in the compact case and will be given in section 4.5. 

The above definitions of orbit complexities follow a topological approach. We 
show that the measure-theoretic setting also provides a natural notion of orbit com- 
plexity lC^{x,T) defined by computable partitions. This kind of orbit complexity 
will be defined almost everywhere and in particular at each /x-random point. For 
this notion the first result in Brudno and White's theorem comes easily. We go 
further in showing: 

Theorem (4.4.2). LetT be an ergodic endomorphism of the computable probability 
space {X, p), 

/C^(x,T) = h^{T) for all ^-random point x. 

We then prove that the two notions of orbit complexity coincide on Martin-L6f 
random points: 

Theorem (5.0.1). LetT be an ergodic endomorphism of the computable probability 
space {X,fi), where X is compact, 

JC^{x,T) = JC{x,T) for all ^-random point x. 

In the topological context, we then consider K,{x,T) and strengthen the second 
part of Brudno's theorem, showing: 
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Theorem (6.3.1). Let T be a computable map on a compact computable metric 
space X , 

sup}C{x,T) = sviplC{x,T) = h{T) 

Remark that this was already imphed by Brudno's theorem, using the variational 
principle: h{T) ~ &\rp{h^{T) : fj, is T-invariant}. Nevertheless, our proof uses 
purely topological and algorithmic arguments and no measure. In particular, it 
does not use the variational principle, and can be thought as an alternative proof 
of it. 

Many of these statements require that the dynamics and the invariant measure 
are computable. 

The first assumption can be easily checked on concrete systems if the dynamics 
is given by a map which is effectively defined. 

The second is more delicate: it is well known that given a map on a metric space, 
there can be a continuous (even infinite dimensional) space of probability measures 
which are invariant for the map, and many of them will be non computable. An 
important part of the theory of dynamical systems is devoted to selecting measures 
which are particularly meaningful. From this point of view, an important class of 
these measures is the class of SRB invariant measures, which are measures being in 
some sense the "physically meaningful ones" (for a survey on this topic see [You02]). 
It can be proved (see [GHR07b] and [GHR07a] and their references e.g.) that in 
several classes of dynamical systems where SRB measures are proved to exist, these 
measures are also computable from our formal point of view, hence providing several 
classes of nontrivial concrete examples where our results can be applied. 

2. Preliminaries 

2.1. Partial recursive functions. The notion of algorithm working on integers 
has been formalized independently by Markov, Church, Turing among others. Each 
constructed model defines a set of partial (not defined everywhere) integer functions 
which can be computed by some effective mechanical or algorithmic (w.r.t. the 
model) procedure. Later, it has been proved that all this models define the same 
class of functions, namely: the set of partial recursive functions. This fact supports a 
working hypothesis known as Church's Thesis, which states that every (intuitively 
formalizable) algorithm is a partial recursive function. It gives the connection 
between the informal notion of algorithm and the formal definition of recursive 
function. 

Let us say then that a recursive function is a function (on integers) that 
can be computed in some effective or algorithmic way. For formal definitions see 
for example, [Rog87]. With this intuitive description it is more or less clear that 
there exists an effective procedure to enumerate the class of all partial recursive 
functions, associating to each of them its Godel number, which is the number 
of the program computing it. Hence there exists a universal recursive function 
: N ^ N satisfying for all e,?i G N, (p„((e,n}) = <Pe('^) where e is the godel 
number of Lp^ and (, ) : — > N is some recursive bijection. In classical recursion 
theory, a set of natural numbers is called recursively enumerable (r. e for short) 
if it is the range of some partial recursive function. That is if there exists an 
algorithm listing the set. We denote by the r.e set associated to t^e- Namely 
Ef, = rang{ipe) = {ipu{{e,n)) : n £ N}. 
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2.2. Algorithms on finite objects. Strictly speaking, algoritlims only work on 
integers. However, when the objects of some class have been identified with integers, 
it makes sense to speak about algorithms acting on these objects. 

Definition 2.2.1. A Numbered Set O is a countable set together with a surjcction 
z^o : N — *■ O called the numbering. We write o„ for i^{n) and call n the name of 

On- 

Of course, the potential of algorithms depends on the choice of the numbering, 
since it determines to what extent an object can be algorithmically recovered from 
its name. If the objects of some collection can be characterized by a finite number 
of integers, then the collection is a numbered set since its objects can be indexed 
using standard recursive bijections from N* to N and from N'' to N, which we will 
both denote (.). 

Examples. (1) Q, with some standard numbering i^q is a numbered set. 

(2) The set of partial recursive functions TZ = {ipe : e G N} is a numbered set, 
Godcl numbers being the names. 

(3) The collection {Ee = rang{ipe) : e e N} of all r.c subsets of N is a numbered 
set. 

Definition 2.2.2. Let O be a numbered set. To any recursive function (y9 : N ^ N 
we associate an algorithm : N — *■ O defined by A^{i) = o^^i). 

Given a total algorithm ^ : N ^ (i.e A — A^ for some total (/s), we say that 
the sequence of finite objects {A{n))n^fi is enumerated by A, or that A is an 
algorithmic enumeration of this sequence. 

2.3. Computability over the reals. 

Definition 2.3.1. Let a; be a real number. We say that: 

• X is lower semi- computable if the set := G N : < x} is r.e, 

• a; is upper semi- computable if the set i? := {i £ N : > x} is r.e, 

• X is computable if it is lower and upper semi-computable. 

Equivalcntly, a real number is computable if and only if there exists an algorith- 
mic enumeration of a sequence of rational numbers converging exponentially fast 
to X. That is: 

Proposition 2.3.1. A real number x is computable if and only if there exists an 
algorithm A : N ^ Q such that \A(i) — < 2^% for all i. 

Definition 2.3.2. Let (x„)„ be a sequence of computable reals. We say that 
the sequence is uniformly computable or that x„ is computable uniformly in 
n if there exists an algorithm ^ : N ^ Q such that for all n and i it holds 

\A{{n,i)) - Xn\ <2-\ 

Uniform sequences of lower (upper) semi-computable reals are defined in the 
same way. 

2.4. Computable Metric Spaces. 

Definition 2.4.1. A computable metric space (CMS) is a triple X = (X,d,S), 
where 

• {X, d) is a separable complete metric space. 

• 5 = (si)igN is a numbered dense subset of X (called ideal points). 

• The real numbers ((i(si, Sj"))(i,j)eN are all computable, uniformly in (i, j). 
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We now recall some important examples of computable metric spaces: 

Examples. (1) the Cantor space (E'*^, d, 5*) with S a finite alphabet and d the 
usual distance. S is the set of ultimately 0-stationary sequences. 

(2) (E", dun , Q") with the euclidean metric and the standard numbering of Q". 

(3) if {Xi,di, Si) and {X2,d2, S2) are two computable metric spaces, the dis- 
tance d{{xi , xi), (yi, 1/2)) = niax(c?i(a;i, ^2(2^2, 2/2)) on the product space 
(Xi X X2, d, Si X Si) makes it a computable metric space. 

For further examples we refer to [Wci93]. 

Let {X, d, S) be a computable metric space. The computable structure of X 
assures that the whole space can be "reached" using algorithmic means. Since 
ideal points (the finite objects of iS) are dense, they can approximate any x at 
any finite precision. Then, x itself can be identified to a sequence of ideal points 
converging to x in an effectively controlled way. Let us say that a sequence of ideal 
points {si^)n is fast if d(sj„,Si„^i) < 2~" for all n. As the space is complete, a 
fast sequence has always a limit x, and d{si^^,x) < 2~("~^^ for all n. 

Definition 2.4.2 (Computable points). A point x G X is said to be computable 

if there exists an algorithm : N — > 5 which enumerates a fast sequence whose 
limit is X. 

As for real numbers we can give the notion of uniform sequence 

Definition 2.4.3. Let (a;„)„ be a sequence of computable points. We say that the 
sequence is uniformly computable or that Xn is computable uniformly in n if 
there exists an algorithm ^ : N — + 5 such that for all n, the sequence {A{{n,i)))i 
is fast and converges to x„. 

The numbered set of ideal points {si)i induces the numbered set of ideal balls 
B := {B{si,qj) : Si & S, qj G Q>o}- We denote by the ideal ball B{si,qj). 

Computability can then be extended from the numbered set B to the space of 
open subsets of X: such an open subset U C X can be identified to a collection of 
ideal balls whose union is U. 

Definition 2.4.4 (R.e open sets). We say that the set ?7 C X is r.e open if there 
is some r.e set E CN such that U = Ui^sBi. 

Remark 2.4. f. Let U he a r.e open set. ft is easy to see that there is an algorithm 
to semi-decide weather some ideal point belongs to U. That is, the algorithm 
will halt on input i iff s; G U. This notion can be extended to any point x in the 
following sense: The algorithm sequentially asks (from an external user) for finite 
approximations of x at required precisions. If a; G C/ the algorithm will eventually 
stop and answer "yes" , if x ^ U then the algorithm will run and ask forever. For 
formal definitions we refer to [HR07]. 

Definition 2.4.5. Let ([/„)„ be a sequence of r.e open sets. We say that the 
sequence is uniformly r.e or that [/„ is r.e open uniformly in n if there exists an 
r.e set E cN such that for all n it holds Un = Uie_E„i?i, where £"„ = {i : {n, i) G E}. 

Examples. (1) If the sequence (?/„)„ is uniformly r.e then the union U„?7„ is a 
r.e open set. 
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(2) The universal recursive function Lpu makes the coUection of all r.e open sets 
(denoted U) a sequence uniformly r.e. Indeed, define E := {(e, ipu{{e, n))) : 
e, n e N}. Then U = {Ue : e e N} where Ue = \Ji^E^Bi. 

(3) The numbered set lA is closed under finite unions and finite intersections. 
Furthermore, these operations arc effective in the following sense: there 
exists recursive functions and iff^ such that for all i, j G N, [/, U Uj = 
UipLi((ij)) and the same holds for (f'^. Equivalently: Ui U Uj is r.e open 
uniformly in . See [HR07]. 

Definition 2.4.6 (Constructive G^-sets). We say that the set 13 C X is a con- 
structive Gs-set if it is the intersection of a sequence of uniformly r.e open sets. 

Let {X, Sx, dx) and (F, Sy, dy) be computable metric spaces with and 
the corresponding numbered sets of r.e open sets. 

Definition 2.4.7 (Computable Functions). A function T : X ^ Y \s said to be 
computable if T~^{Bn) is r.e open uniformly in n. 

Remark 2.4.2. We remark that this definition implies that the preimage of a uniform 
sequence of r.e. open sets is a uniform sequence of r.e. open sets. This could be an 
alternative definition of computable function. 

It follows that computable functions are continuous. Since we will work with 
functions which are not necessarily continuous everywhere, we shall consider func- 
tions which are computable on some subset of X. More precisely, a function T is 
said to be computable on D {D C X) if there is a uniform sequence {U^)n of r.e 
open subsets of X such it holds T^^(Bn) D D ^ fl D for the uniform sequence 
of ideal balls i3„. D is called the domain of computability of T. 
remarks: 

• Since ideal balls generate the topology, a function is computable iff T~^{B^) 
is r.e open uniformly in n (or the intersection of D with a uniformly r.e open 
set). 

• If T is computable then the images of ideal points can be uniformly com- 
puted, that is: T{sf) is a computable point, uniformly in i. 

• More generally, if T is computable then there exists an algorithm which 
computes the image T[x) of any x in the following sense: the user enters 
some rational e to the algorithm which, after asking finitely many times the 
user for finite approximations of x, halts outputting a finite approximation 
of T{x) up to e. 

• The distance function d : X x X ^M. \s a. computable function. 

2.5. Computable Probability Spaces (CPS). When X is a computable metric 
space, the space of probability measures over X, denoted by AA{X), can be endowed 
with a structure of computable metric space (this will be defined below, for more 
details, see [Gac05, HR07]). Then a computable measure can be defined as a 
computable point of M.{X). 

Some prerequisites from measure theory: We say that /i„ converge weakly to /i 
and write ^ /i if 

^ /i iff Mil/ ^ m/ for all real continuous bounded / (2) 
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where fif stands for J fd^i. Let us recall the Portmanteau theorem. We say that a 
Borel set A is ^-continuous if = 0, where dA ^ Ar\X\A\s the boundary 

of A. 

Theorem 2.5.1 (Portmanteau theorem). Let fin, fJ- be Borel probability measures 
on a separable metric space {X, d) . The following are equivalent: 

(1) fin converges weakly to fi, 

(2) limsup„ lin{F) < fJ,{F) for all closed sets F, 

(3) liminf„ /i„(G') > niG) for all open sets G, 

(4) liuin fJ-niA) ~ tJ-{A) for all fi-continuity sets A. 

This theorem easily implies the following: when {X, d) is a separable metric 
space, weak convergence can be proved using the following criterion: 

Proposition 2.5.1. Let A be a countable basis of the topology which is closed under 
the formation of finite unions. If iJLn{A) — > /x(A) for every A ^ A, then /x„ converge 
weakly to fi. 

Let us introduce on M{X) the structure of a computable metric space. Let us 
endow Ai{X) with the weak topology, which is the topology of weak convergence. 
As X is separable and complete, so is M{X). Let D C A4{X) be the set of those 
probability measures that are concentrated in finitely many points of S and assign 
rational values to them. It can be shown that this is a dense subset ([BilG8]). 

We consider the Prokhorov metric p on A4{X) defined by: 

p{n, v) := inf{e G IR+ : p.{A) < iy{A^) + e for every Borel set A}. 

where A" = {x : d{x, A) < e}. 

This metric induces the weak topology on /iA{X). Furthermore, it can be shown 
that the triple {M{X), D, p) is a computable metric space (see [Gac05], [HR07]). 

Definition 2.5.1. A measure p. is computable if there is an algorithmic enumer- 
ation of a fast sequence of ideal measures {pn)neN C D converging to p in the 
Prokhorov metric and hence, in the weak topology. 

The following theorem gives a characterization for the computability of measures 
in terms of the computability of the measure of sets (for a proof see [HR07]): 

Theorem 2.5.2. A measure p £ Ai{X) is computable if and only if the mea- 
sure p{Bi-^ U . . . U of finite unions of ideal open balls is lower-semi-computable 
uniformly in {ii, . . . , i^) . 

Definition 2.5.2. A Computable Probability Space (CPS) is a pair {XtP) 
where X is a, computable metric space and /i is a computable Borel probability 
measure on X. 

Definition 2.5.3. Let {X^p) and {y,v) be two computable probability spaces. 
A morphism from (A", p) to (3^, v) is a measure-preserving function F : X ^ Y 
which is computable on a constructive G^-set of /i-measure one. 

We recall that F is measure-preserving if v{A) = p{F~^{A)) for every Borel set 
A. Computable probability structures can be easily transferred: 

Proposition 2.5.2. Let (X, p) be a computable probability space, y be a computable 
metric space and F : X Y a function which is computable on a constructive Gs - 
set of p-measure one. Then the induced measure pp on Y defined by pf{A) = 
p{F^^{Ay) is computable and F is a morphism of computable probability spaces. 
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2.6. Algorithmic randomness. Now we consider a generalization of Martin-Lof 
tests to computable probability spaces. Let {X, ^) be a computable probability 
space. 

Definition 2.6.1. A Martin-Lof fi-Test is a sequence {Un)n£n of uniformly r.e 
open sets which satisfy /i(C/„) < 2^" for all n. Any subset of p|„ Un is called an 
effective fi-null set. 

Definition 2.6.2. A point x G X is called fi-random if x is contained in no 
effective //-null set. The set of /i-random points is denoted i?^. 

Note that ^{R^j) = 1. The following is the generalization for metric spaces of 
a classical result in Cantor space due to Martin-Lof. It says that the set of non- 
random points is not only a null set but an effective null set. For a proof see 



Theorem 2.6.1. The union of all effective fi-null sets, denoted by Mp,, is again an 
effective fi-null set. 

Thus, there is a single Martin-Lof test (often called universal) which tests non- 



We will need the following results, also taken from [HR07]. 

Lemma 2.6.1. Every fi-random point is in every r.e open set of full measure. 

Proposition 2.6.1 (Morphisms of CPS preserve randomness). Let F be a mor- 
phism of computable probability spaces (X,^) and (3^, i/). Then every fi-random 
point X is in the domain of computability of F and F{x) is v-random. 

2.7. Kolmogorov complexity. The idea is to define, for a finite object, the min- 
imal amount of algorithmic information from which the object can be recovered. 
That is, the length of the shortest description (code) of the object. Since this short- 
est description is supposed to contain all necessary information to reconstruct in an 
algorithmic way the coded finite object, the Kolmogorov Complexity is also called 
Algorithmic Information Content. For a complete introduction to Kolmogorov com- 
plexity we refer to a standard text [LV93] . 

Let E* and Y,^ be the sets of finite and infinite words (over the finite alphabet 
S) respectively. A word w e E* defines the cylinder [w] C of all possible 
continuations of w. A set Z) = {wi,W2, ...} C E* defines an open set [D] = Ui[wi] C 
E'^. D is called prefix-free if no word of D is prefix of another one, that is if the 
cylinders [wi] are pairwise disjoint. 

Let X be E* or N or N*. 

Definition 2.7.1. An interpreter is a partial recursive function Lp : {0, 1}* X 
which has a prefix-free domain. 

Definition 2.7.2. Let / : {0, 1}* ^ A be an interpreter. The complexity (or 
Information Content) Kj{x) of .t G A is defined to be 



It turns out that there exists an algorithmic enumeration of all interpreters, 
which entails the existence of a universal interpreter U which is asymptotically 
optimal in the sense that the invariance theorem holds: 



[HR07]. 



randomness, and = N^. 




\p\ if p is a shortest input such that I{p) 
oo if there is no p such that I{jp) = x 



— X 
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Theorem 2.7.1 (Invariance theorem). For all interpreter I there exists c/ G N 
such that for all x ^ X we have Kiriw) < Ki{x) + c/. 

We fix a universal interpreter U and we let K(x) = Kij(x). 

2.7.1. Estimates. Let us recall some simple estimates of complexity. Let f,g be 
real-valued functions. We say that g additively dominates f and write / < g if 
there is a constant c such that f < g + c. As codes are always binary words, we 
use base-2 logarithms, which we denote by log. We define J{x) = x + 21og(x + 1) 
for a; > 0. 

For n e N, K{n) < J(logn). For m, . . . , rife g N, K{ni, ...,nk)< K{ni) + ...+ 
K{nk). The following property is a version of a result attributed to Kolmogorov, 
stated in terms of prefix complexity instead of plain complexity. 

Proposition 2.7.1. Let E <ZH x X be a r.e. set such that £"„ = {x : {n,x) £ E} 
is finite for all n. Then for (s, n) with s £ En, 

Kis) < Jilog\En\) + K{n) 

Proposition 2.7.2. Let ji be a computable measure on E***. For all G E*, 

K{w)< - log ^Ji{[w\) + K{\w\) 

Theorem 2.7.2 (Coding theorem). Let P : X ^ be a lower semi- computable 
function such that ^"(2;) < 1. Then K{x) < — logP(a;), i.e. there is a constant 
c such that K{x) < — \ogP{x) + c for all x £ X . 

Moreover, 2"^'^^ < 1 as it is the Lebesgue measure of the domain of the 
universal interpreter U . There is a relation between Kolmogorov complexity and 
randomness, initial segments of random infinite strings being maximally complex. 

Theorem 2.7.3 (Chaitin, Levin). Let n be a computable measure. Then uj G 
is a ^-random sequence if and only if 3m Vn K{uji;n) > — log/i[cji:„] — m. 

The minimal such m, defined by dn{uj) := sup„{— log/^[a'i:„] — K(uji;n)} and 
called the randomness deficiency of oj w.r.t /i, is not only finite almost every- 
where: it has finite mean, that is J d^{u!)dfi < 1. For a proof see [LV93]. 

3. Effective symbolic dynamics and statistics of random points 

Let (X, /z) be a computable probability space and let i?^ be the set of random 
points. The aim of this section is to study the set from a dynamical point of 
view. That is, we will put a dynamic T on {X, /i) (an endomorphism of computable 
probability space), and look at the abilities of random points (which are a priori 
independent of T) to describe the statistical properties of T. 

We recall that a Borel set A is called T -invariant if T^^{A) ~ y4(mod 0) and 
that the transformation T is said to be ergodic if every T-invariant set has measure 
or 1. 

3.1. Symbolic dynamics of random points. Let T be an endomorphism of the 
(Borel) probability space (X, /i) . In the classical construction, one considers access 
to the system given by a finite measurable partition, that is a finite collection of 
pairwise disjoint Borel sets V = {pi,...,pfe} such that /z(UiPi) = 1. Then, to 
{X,iJL,T) is associated a symbolic dynamical system {X-p,a) (called the symbolic 
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model of (X, T, V)). The set Xv is a subset of {1,2,..., k}^. To a point x e X 
corresponds an infinite sequence to = (wi)igN = 4>v{x) defined by: 

The transformation a : X-p X-p is the shift defined by (T((a;i)igN) = (a;i+i)igN- 
As is a measurable partition, the map (j)-p is measurable and then the measure 

/J, induces the measure (on the associated symbolic model) defined by ^j,'p{B) = 

IJ,{(I)^^ (B)) for all measurable B C X-p. 

The requirement of 0-p being measurable makes the symbolic model appropriate 

from the measure-theoretic view point, but is not enough to have a symbolic model 

compatible with the computational approach: 

Definition 3.1.1. Let T be an endomorphism of the computable probability space 
{X,fi) and V = {pi . . . ,Pk} a finite measurable partition. The associated symbolic 
model {Xp,iJ,p,a) is said to be an effective symbolic model if the map (f>p : 
X ^ {1, . . . , k}^ is a morphism of CPS (here the space {1, . . . , k}^ is endowed with 
the standard computable structure). 

The sets pi are called the atoms of P and we denote by P{x) the atom containing 
X (if there is one) . Observe that (jyp is computable on its domain only if the atoms 
are open r.e sets (in the domain): 

Definition 3.1.2 (Computable partitions). A measurable partition V is said to be 
a computable partition if its atoms are r.e open sets. 

Conversely: 

Tiieorem 3.1.1. Let T be an endomorphism of the CPS {X, /i) and P = {pi . . . ,pk} 
a finite computable partition. Then the associated symbolic model is effective. 

Proof. Let D be the domain of computability of T (it is a full-measure constructive 
Gs). Define the set 

x'^ = Dnf] r-"(piu...upfc) 

nGN 

is a full-measure constructive G^-sct: indeed, as pi U . . . U p/c is r.e. and T is 
computable on D there are uniformly r.e. open sets J7„ such that D n r^"(pi U 
. . .Upk) = D nUn, so A"'' = Z? n P|„ Un. As T is measure-preserving, all Un have 
measure one. 

Now, AT'P n [io, . . . , in] = AT'P n p.,„ n T-^p,^ n . . . n r~"pi„ . This proves that 
<j>P is computable over X^ . Proposition 2.5.2 allows to conclude. □ 

After the definition an important question is: are there computable partitions? 
the answer depends on the existence of open r.e sets with a zero-measure boundary. 

Definition 3.1.3. A set A is said to be almost decidable if there are two r.e 
open sets U and V such that: 

UciA, ycA", ijl{U) + pl{V) = I 

remarks: 

• a set is almost decidable if and only if its complement is almost decidable, 

• an almost decidable set is always a continuity set, 

• a /i-continuity ideal ball is always almost decidable. 
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• unless the space is disconnected (i.e. has non-trivial clopen subsets), no 
set can be decidable, i.e. semi-decidable (r.e) and with a semi-decidable 
complement (such a set must be clopen^). Instead, a set can be decidable 
with probability 1: there is an algorithm which decides if a point belongs 
to the set or not, for almost every point. That is why we call it almost 
decidable. 

Ignoring computability, the existence of open /^-continuity sets directly follows 
from the fact that the collection of open sets is uncountable and fi is finite. The 
problem in the computable setting is that there are only countable many open r.e 
sets. Fortunately, there still always exists a basis of almost decidable balls. This 
result, first obtained in [IIR07] with other techniques, will be used many times in 
the sequel, in particular it directly implies the existence of computable partitions. 
For completeness we present a different, self-contained proof. 

Theorem 3.1.2. There is a family of uniformly computable reals (r^)j_„gN such 
that for all i, {r^ : n G N} is dense in ]SJ^ and such that for every i,n, the ball 
B[si,r\^) is almost decidable. 

Proof. Let Si be an ideal point. Put It^j.k) ~ ['Zji'Zfc] with qj,qk positive rational 
numbers. We show that for every n = (j, k) we can compute, uniformly in n, a 
real r]^ g /„ for which fi{dB{si,r'!^)) = 0. First observe that for a closed interval 
/ = [a,b] {a,b G Q), the complement of Bj = B{si,b)/B{si,a), is r.e open. Then 
by corollary 2.5.2, its measure is lower semi-computable and then we can semi- 
decide for a given rational q the relation fi{Bi) < q. The algorithm computing r'^ 
enumerates a sequence of nested closed intervals {Jk)keN whose length tends to 0, 
with Jo = /„, and such that for aU k, fi{BjJ < 2"''+^. Then {r-JJ = rik>iJk- It 
works as follows: 

In stage fc + 1 (the interval Jk ~ [a,b] has already been found), put m = and 
test in parallel f^{B[a,a+m]) < 2^'^ and n{B[ij_m^b]) < 2^^^. Since ix{Bj^) < 2"'^+^, 
one of the tests must stop, and then provides the "good" interval Jk+i for which 
the condition holds. □ 

We denote by i?^*^") the almost decidable ball B{si^r\^). 

The family : i,n e N} is a basis for the topology. It is even effectively 

equivalent to the basis of ideal balls : every ideal ball can be expressed as a r.e. 
union of almost decidable balls, and vice- versa. 

We finish presenting some results that will be needed in the next subsection. 

Corollary 3.1.1. On every computable probability space, there exists a family of 
uniformly computable partitions which generates the Borel a-algebra. 

Proof. Take = {-B(si, r'J, X \ i3(si,r^)} where B is the closed ball: as the 

almost decidable balls form a basis of the topology, the cr-algebra generated by the 
Pk is the Borel cr-field. □ 

Proposition 3.1.1. If A is almost decidable then ^{A) is a computable real number. 



In Cantor space for example (which is totally disconnected), every cylinder (ball) is a decidable 
set. Indeed, deciding if some infinite sequence belongs to some cylinder reduces to a finite pattern- 
matching. 
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Proof. Since U and V are r.e open, by theorem 2.5.2 their measures arc lowcr-semi- 
computable. As fJ.{U) + fi{V) = 1, their measures are also upper-semi-computable. 

□ 

The following regards the computability of inducing a measure in a subset and 
will be used in the proof of prop. 3.2.1 

Proposition 3.1.2. Let ^ be a computable measure and A an almost decidable 
subset of X . Then the induced measure fiAi-) = /^(-l^) computable. Furthermore, 

Proof let W = U . . . U i3„^ be a finite union of ideal balls. ^a(M^) = IJ-iW fl 
A)l ^{A) = fi{W n U)/fi{A). W n [/ is a r.e open set, so its measure is lower semi- 
computable. As IJ,{A) is computable, ij,a{W) is lower semi-computable. Note that 
everything is uniform in (ni, . . . The result follows from theorem 2.5.2. 

Let U and V as in the definition of an almost decidable set. First note that 
Rf,r\A ^ R^nU, as Rf, CUUVhy lemma 2.6.1. Again by lemma 2.6.1, i?^^ C U, 
and as fj,A < JI(a)1^' every /i-effective null set is also a /iyi-effective null set, so 
RfiA ^ R/j.- Hence, we have i?^^ C R^nU. 

Conversely, i?^^ being a /x^-effective null set, its intersection with C/ is a /i- 
effective null set, by definition of fiA- So i?^^ n [/ C _RJ^, which is equivalent to 

R^^nucR^^. □ 

3.2. Some statistical properties of random points. With the tools developed 
so far, it is possible to translate many results of the form 

tx{x : P{x)} = 1, 
with P some predicate, into an "individual" result of the form: 

"If X is /i-random, then P{xy' . 
In this section we give two examples: recurrence and statistical typicality. 

Definition 3.2.1. Let X be a metric space. A point x ^ X is said to be recurrent 

for a transformation T : X ^ X , ii liminf„ d{x, T".t) = 0. 

Proposition 3.2.1 (Random points are recurrent). Let [X^pL) be a computable 
probability space. If x is pL-random, then it is recurrent with respect to every mea- 
sure preserving endomorphism T on {X,fi). 

Proof, take x G i?^ and B an almost decidable neighborhood of x. Then fi{B) > 
and there is a r.e open set U such that: 

y T-"B = Ur\D 

n>l 

where D is the domain of computability of T. By the Poincare recurrence theorem, 
this set has full measure for ^si-) = t'-i-lB). By proposition 3.1.2, x G Rfig^ by 
lemma 2.6.1, x is in U. □ 

We now prove that random points satisfy a stronger property to be used in the 
sequel: statistical typicality. Let us then introduce this concept. 

Let X be a metric space and T be a continuous transformation on X. Let Cb{X) 
be the space of bounded real-valued continuous functions on X. For / S Cb{X) 
define: 
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_j n — 1 

7{x) -.^ lim - V f{Px) (3) 
i=o 

at the points x where this hmit exists. We recall that a point x is called generic 
for T if f{x) is defined for every / G Cb{X). 

Every generic point x generates a probability measure which is invariant for 
T, dually defined by: 

/ fd^Jix = l{x) for all / G Cb{X)- (4) 

In other words, x is generic if the measure f„ ^ X]j<n ^t^x converges weakly 
to where 5y is the Dirac probability measure concentrated on y. Let /i be an 
crgodic measure for T. A generic point x is said to be fi-typical if jj^x = /i. The 
well-known Birkhoff ergodic theorem says that for each ergodic measure ^, the set 
of /i-typical points has /i-measure one. 

From a statistical point of view, ju-typical points are those whose orbits reproduce 
the main statistical features of /i (in particular they are a total measure set), hence 
in some sense they are random for the dynamic. 

What algorithmically random points have to do with dynamically random points? 

This problem has already been studied by V'yugin ([V"> !)7]) in the particular case 
of the Cantor space and for computable observables. We prove a general version 
which applies to computable dynamics on any computable probability space, for 
any bounded continuous (not necessarily computable) observable. The strategy is 
simple: we use computable partitions to construct effective symbolic models and 
use the following particular case of V'yugin's main theorem. 

Lemma 3.2.1. Let jj. be a computable shift-invariant ergodic measure on the Cantor 
space {0, 1}'^. Then for each fi-random sequence lu: 

1 " 

lim-^u;, = M([l]) (5) 

i=0 

We are now able to prove: 

Theorem 3.2.1. Let {X, fi) be a computable probability space. Then each ^-random 
point X is fi-typical for every ergodic endomorphism T . 

We remark that the theorem holds uniformly for all bounded continuous observ- 
ables and all ergodic endomorphisms. 

Proof. Let /a be the characteristic function of the set A. First, let us show that if 
A is an almost decidable set then for all /x-random point x: 

n 

\\m-y^fAoT{x)=^l{A) (6) 

Indeed, consider the computable partition defined by V := {U, V} with U and V 
as in definition 3.1.3 and the associated symbolic model [X-p, a, fi-p)- By proposition 
3.1.1, (pvix) is a well defined /ip-random infinite sequence, so lemma 3.2.1 applies 
and gives (6). This can be reformulated as the convergence of Vn{A) to ^J^{A). Now, 



THE DYNAMICS OF ALGORITHMICALLY RANDOM POINTS 



15 



the collection of almost decidable sets satisfies proposition 2.5. f, so Un converges 
weakly to ix: x is /i-typical. □ 

4. Measure-theoretic entropies 

Suppose discrete objects (symbolic strings for instance) are produced by some 
source. The tendency of the source toward producing such object more than such 
other can be modeled by a probability distribution, which gives more information 
than the crude set of possible outcomes. The Shannon entropy of the probabilistic 
source measures the degree of uncertainty that lasts when taking the probability 
distribution into account. 

Any ergodic dynamical system [X, T, /i) can be seen as a source of outputs. 
Kolmogorov and Sinai adapted Shannon's theory to dynamical systems in order 
to measure the degree of unpredictability or chaoticity of an ergodic system. The 
first step consists in discretizing the space X using finite partitions. Let ^ = 
{Ci, . . . , C„} be a finite measurable partition of X. Then let T~^^ be the partition 
whose atoms are the pre- images T~^Ci. Then let 

in = C V T-^i V T"2^ V ... V T-("-i'C 
be the partition given by the sets of the form 

a„ nT-ic.i n...nr-("-i)a„_,, 

varying Ci- among all the atoms of i. Knowing which atom ^„ a point x belongs 
to comes to knowing which atoms of the partition ^ the orbit of x visits up to time 
n — 1. 

The measure-theoretical entropy of the system w.r.t the partition ^ can then be 
thought as the rate (per time unit) of gained information (or removed uncertainty) 
when observations of the type "r"(x) S C^" are performed. This is of great im- 
portance when classifying dynamical systems: it is a measure-theoretical invariant, 
which enables one to distinguish non-isomorphic systems. 

We briefly recall the definition. For more details, we refer the reader to [BiKiT)], 
[Wal82], [Pet83], [HK95]. 

4.1. Entropy with Shannon information. Given a partition ^ and a point x, 
i{x) denotes the atom of the partition x belongs to. Let us consider the Shan- 
non information function relative to the partition ^„ (the information which is 
gained by observing that x G £,n{x)), 

and its mean, the entropy of the partition 

The measure-theoretical or Kolmogorov- Sinai' entropy of T relative to the 
partition ^ is defined as: 

h^{T,0 = lim -H^ii„). 

(which exists and is an infimum, since the sequence i?p(^n)n is sub-additive). With 
the Shannon information function, it is possible to define a kind of point-wise notion 



16 



STEFANO GALATOLO, MATHIEU HOYRUP, AND CRISTOBAL ROJAS 



of entropy with respect to a partition ^ : 

lim sup -/^(x|^„). 

This local entropy is related to the global entropy of the system by the celebrated 
Shannon-McMillan-Breiman theorem: 

Theorem (Shannon-McMillan-Breiman). LetT he an ergodic endomorphism of the 
probability space (X,^,^) and ^ a finite measurable partition. Then for fj,-almost 
every x, 

lim i/^(x|C„) = /i^(r,0. (7) 
The convergence also holds in L^{X,£§,^). 

Now we suppress the partition-dependency: the Kolmogorov- Sinai entropy 
of (X, T, Ai) is 

h^{T) :~ s\xp{h^(T,^) : ^ finite measurable partition} 

We recall the following two results that we will need later. The first proposition 
follows directly from the definitions. 

Proposition 4.1.1. // (E'*', cr) is the symbolic model associated to (X, /i,T, ^) 
then hf^{T,^) = h^j_^{a). 

The next proposition is taken from [Pet83]: 

Proposition 4.1.2. // (^i)^^^ is a family of finite measurable partitions which 
generates the Borel u -field up to sets of measure 0, then h^(T) = supjft-^(r, V 
...V6). 

4.2. Entropy with Kolmogorov information. In this section, T is an endo- 
morphism of the computable probability space {X^^) and ^ = {Ci, . . . ,Cfe} is a 
computable partition. Let (E^, /i^, a) be the effective symbolic model of {X, /x, T, ^) 
where E = {1, . . . , A:} (see section 3.1). 

Kolmogorov introduced his algorithmic information content (also called Kol- 
mogorov complexity) as a quantity of information, on the same level as Shannon 
information. When the measure, the transformation and the partition are com- 
putable, it makes sense to define the algorithmic equivalents of the notions defined 
above. It turns out that the two points of view are strongly related. 

An atom C of the partition ^„ can then be seen as a word of length n on the 
alphabet E, which allows one to consider its Kolmogorov complexity K{C). For 
those points whose all iterates are covered by ^ (they form a constructive dense Gs 
of full measure), we define the Kolmogorov information function relative to 
the partition ^„: 

which is independent of ^. We can then define algorithmic entropy of the par- 
tition ^„ as the mean of X: 

n,.i^n):= f x(.|c„)d/.= I'icmc). 

We also define a local notion of algorithmic entropy, which we call symbolic orbit 
complexity: 
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Definition 4.2.1 (Symbolic orbit complexity). Let T be an endomorphism of the 
computable probability space (X, /i). For any finite computable partition ^, we 
define /C^(.t,T|^) := limsup„ iZ(a;|^„). Then, we can suppress the dependence on 
^ by taking the supremum over all computable partitions: 

lCi^i{x,T) :— sup{/C^(x, T|^) : ^ computable partition} 

As there are only countably many computable partitions. ]C^{x,T) is defined 
almost everywhere (at least on random points). 

The quantity /C^ (x, T|^) was introduced by Brudno in [Bru83] without any com- 
putability restriction on the space, the measure nor the transformation. He proved: 

Theorem 4.2.1 (Brudno). ICf^{x,T\^) ~ /i^(r, ^) for ^-almost every point. 

Remark 4.2.1. Already Brudno remarked that if x has not an eventually periodic 
orbit, by taking the supremum of lC^{x,T\£) over all - not necessarily computable 
- finite partitions ^ generally gives an infinite quantity, that is why Brudno did not 
go further (he did not have a computable structure at his disposal), and proposed 
a topological definition using open covers instead of partitions. 

We will compare /C^ and Brudno orbit complexity in section 4.5, we now show 
that the hypothesis of definition 4.2.1 enables one to derive Brudno's theorem in a 
rather simple manner. 

The theory of randomness and Kolmogorov complexity on the space of symbolic 
sequences provides powerful results (theorem 2.7.3 and proposition 2.7.2) which 
enable to relate the algorithmic entropies and to the Shannon entropies 
and (inequalities (9), (11))- We recall these two results: if is endowed with 
a computable probability measure v, then for all lo S S^, 

-\ogv[ujQ, , n-i] ~ d^{io) < K{uJo..n~i) < - \ogiy[uin..n-i] + K (n) (8) 

where di, is the deficiency of randomness, which satisfies J^^m di,dv < 1 and is finite 
exactly on Martin-L6f random sequences (the constant in < does not depend on 
uj and n, see section 2.7.1). 

4.3. Equivalence between local entropies. Applying (8) to v ^ ji^ directly 
gives: 

I^.{Mn)-d^ocp^ < X(.|e„) < I^{.\^n) + K{n) (9) 

where it is defined (almost everywhere, at least on random points). Every /.i-Martin- 
Lof random point x is mapped by (j)^ on a ^^-Martin-Lof random sequence (see 
proposition 2.6.1), whose randomness deficiency is finite. It then follows that the 
local entropies using Shannon information and Kolmogorov information coincide 
on /i-random points: 

Proposition 4.3.1. 

/Cp(a;,T|^) = limsup — /(a;|^„) for every ^-Martin-Lof random point x (10) 
n ri 

This equality together with the Shannon-McMillan-Breiman theorem (7) give 
directly Brudno's theorem (theorem 4.2.1). 



18 



STEFANO GALATOLO, MATHIEU HOYRUP, AND CRISTOBAL ROJAS 



Remark 4.3.1 (Equivalence between global entropies). Now, the Kolmogorov-Sinai 
entropy, originally expressed using Shannon entropy, can be expressed using algo- 
rithmic entropy. Taking the mean in (9), one obtains: 



i?M(CrO-l < n^^iU < H^{^r.) + K{n) (11) 

So, 

hJT\0 = lim = lim 

n n n n 

As the collection of computable partitions is generating (see corollary 3.1.1), the 
Kolmogorov-Sinai' entropy of {X, fi, T) can be characterized by: 

hnCT) = sup i lim '^^^^"'^ : ^ finite computable partition 
[ n n 

It then follows that K.^ {x, T) = h^ {T) for //-almost every x. We now strengthen 
this, proving that it holds for all Martin-L6f random points. 

4.4. Orbit complexity vs entropy. On the Cantor space, V'yugin ([V"y97]) 
and later Nakamura ([XnkO'i]) proved a slightly weaker version of the Shannon- 
McMillan-Breiman for Martin-L6f random sequences. In particular, we will use: 

Theorem 4.4.1 (V'yugin). Let fj. be a computable shift-invariant ergodic measure 
on S^. Then, for any ji-Martin-Lof random sequence uj, 

limsup--log/i([tJo..«-i]) = h^{(T). 

n — >QO ri 

Note that it is not known yet if the limit exists for all random sequences. 
Using effective symbolic models, this can be easily extended to any computable 
probability space. 

Corollary 4.4.1 (Shannon-McMillan-Breiman for random points). Let T be an er- 
godic endomorphism of the computable probability space {X,fi), and a computable 
partition. For every ^-Martin-Lof random point x, 

limsup--log/i(^„(x-)) = hi,{T,^). 

n — *oo ri 

Proof. Since ^ is computable, the symbolic model [X^, a) is effective. Every fi- 
Martin-L6f random point x is mapped to a /ij-Martin-L6f random sequence lo, for 
which the preceding theorem holds. Using the facts that fJ-{S,n{x)) = /i^([tJo..n-i]) 
and h^{T,^) ~ h^^{a) allows to conclude. □ 

Finally, this implies our first announced result: 

Theorem 4.4.2. Let T be an ergodic endomorphism of the computable probability 
space {X,fi). For every fi-Martin-L6f random point x: 

/c^(x,T) = /i^(r). 

Proof. We combine equality (10) and corollary 4.4.1: for every random point x, 
ICfj,{x,T\^) = limsup„ ^/^(a;|^„) = h^{T,^). Since the collection of all computable 
partitions generates the Borel cr-field (corollary 3.1.1), /C^(a;,r) = sup{/ip(T, ^) : 
^ computable partition} = /i^i(r) (proposition 4.1.2). □ 
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4.5. Orbit complexity. In this section, {X,d,S) is a computable metric space 
and T : X ^ X a transformation (for the moment, no continuity or computability 
assumption is put on T). We will consider a notion of orbit complexity which quan- 
tifies the algorithmic information needed to describe the orbit of x with finite but 
arbitrarily accurate precision. The definition we will give coincide with Brudno's 
original definition on compact spaces (see [GaKJO]). 

Given e > and ti £ N, the algorithmic information that is needed to list a 
sequence of ideal points which follows the orbit of x for n steps at a distance less 
than e is: 

ICn{x, T, e) := inm{K{io, i„_i) : d{si^ , T^x) < e for j = 0, . . . , n - 1} 

where K is the self-delimiting Kolmogorov complexity. 

We then define the maximal and minimal growth-rates of this quantity: 

}C{x,T,e) := limsup — /C„(x, T, e) 

n — >OG 

}C{x,T,e) := liminf — /C„(x, T, e). 

n — >OG ft 

As e tends to 0, these quantities increase (or at least do not decrease), hence 
they have limits (which can be infinite). 

Definition 4.5.1. The upper and lower orbit complexities of x under T are 
defined by: 

K{x,T) := lim ]C{x,T,t) 
lC{x,T) := lim ]C{x,T,e). 

£^0+ 

Remark 4.5.1. If T is computable, and assuming that e takes only rational val- 
ues, the n first iterates of x could be e-shadowed by the orbit of a single ideal 
point instead of a pseudo-orbit of n ideal points. Actually it is easy to see that 
it gives the same quantities lC{x,T,e) and lC{x,T,e): let IC'„{x,T,e) = mm{K{i) : 
d{T^Si,T^x) < e for j < n}, one has: 

IC'„{x,T,2e) < IC„{x,T,e)+K{e) 
/C„(x,r,6) < /C;(x,T,e/2)+if(n,e) 

Indeed, from e and iq, . . . , in-i some ideal point can be algorithmically found in the 
constructive open set B{sig, e) fl . . . fl T~*^"~^^i?(si^_^, e), uniformly in iq, ■ ■ ■ , in-i- 
Its n first iterates 2e-shadow the orbit of x, which proves the first inequality. For 
the second inequality, some io,...,in-i can be algorithmically found from n, e, 
and a point Si whose n first iterates e/2-shadow the orbit of x, taking any s^. S 
B(r^s„e/2). 

Remark 4.5.2. Under the same assumptions, one could define K{Bn{si, e)) to be 
K{i, n, e), and replace K(i) by K{Bn{si, e)) in the definition of /C^(a;, T, e), without 
changing the quantities IC{x, T, e) and IC{x, T, e). Indeed, 



K{i) < K{B„{s^, e)) < K{i) + K{n) + K{e) 
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5. Equivalence of the two notions of orbit complexity for random 

POINTS 

We now prove: 

Theorem 5.0.1. Let T be an ergodic endomorphism of the computable probability 
space {X,fi), where X is compact. Then for every Martin-Lof random point x, 

IC{x,T) = K.^{x,T). 

Proof of IC{x,T) < JCfj,{x,T). Let e > 0. Choose a computable partition ^ of diam- 
eter < e (this is why we require X to be compact). To every ceU of associate an 
ideal point which is inside (as ^ is computable, this can be done in a computable 
way, but we actually do not need that). The translation of symbolic sequences in 
sequences of ideal points through this finite dictionary is constructive, and trans- 
forms the symbolic orbit of a point x into a sequence of ideal points which is e-close 
to the orbit of x. So IC{x,T,e) < /C^(x,T|^). The inequality follows letting e tend 
to 0. □ 

To prove the other inequality, we recall some technical stuff. The self-delimiting 
Kolmogorov complexity of natural numbers k > 1 satisfies 

K{k) < f{k) 

where f{x) = logx + 1 + 21og(loga; + 1) for all a; G M, x > 1. / is a concave 
increasing function and x ^ xf{l/x) is an increasing function on ]0, 1] which tends 
to as X —> 0. 

We recall that for finite sequences of natural numbers (fci, . . . , A:„), one has 
K(fci,...,fc„) <if(fci) + ... + if(fc„) 

as the shortest descriptions for fci , . . . , fc„ can be extracted from their concatena- 
tion (this is one reason to use the self-delimiting complexity instead of the plain 
complexity) . 

Lemma 5.0.1. Let 12 be a finite alphabet andn G N. Letu,v € S" andO < a < 1/2 
such that the density of the set of positions where u and v differ is less than a, that 
is: 

< n : u, V,} < a < 1/2 

n 

Then \^K{u) — ^K{v)\ < af{l/a) + -^ where c is a constant independent ofu,v 
and n. 

Proof. Let (ii, . . . , ip) be the ordered sequence of indices where u and v differ. By 
hypothesis, p/n < a. Put ki = ii and kj = ij — ij-i for 2 < j < p. 

We now show that u can be recovered from v and roughly af{l/a)n bits more. 
Indeed u can be computed from (w, fci, . . . , fcp), constructing the string which coin- 
cides with V everywhere but at positions ki, ki + k2, . ■ . , ki + . . . + kp, so K{u) < 
^ K{v) + K{k,) + ... + K{kp) < K{v) + f{k^) + ... + f{kp). 

Now, as / is a concave increasing function, one has: 
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As a result, 

n n n \p J n 

where c is some constant independent of u^v,n,p. As p/n < a < 1/2 and x t— > 
xf{l/x) is increasing for x < 1/2, one has: 

-K{u) < -K{v) + af{l/a) + - 
n n n 

Switching u and v gives the resuh (c may be changed). □ 

We are now able to prove the other inequality. 

Proof of ICfj,{x,T) < IC{x,T). Fix some computable partition ^. We show that for 
any (3 > there is some e > such that for every Martin-Lof random point x, 
ICfj,{x,T\^) < /C(.T,T, e) + (3. As IC{x,T,e) increases as e ^ 0+ and /3 is arbitrary, 
the inequality follows. 

First take a < 1/2 such that af{l/a) < l3, and remark that 

hm M (WF) = M^O = 

£-+0+ V / 

Hence there is some e such that /i ^(S^)^*^^ < a. From a sequence of ideal points 
we will reconstruct the symbolic orbit of a random point with a density of errors 
less than a. Lemma 5.0.1 will then allow to conclude. 

We define an algorithm A{e,io-, ■ ■ ■ ,in-i) with e G 'Q>o and io,...,jn-i G N 
which outputs a word aq . . . a„_i on the alphabet ^. To compute Oj, A semi-decides 
in a dovetail picture: 

• Si - e C for every C G ^, 

• s G C for every s G B{si. , e) and every C G ^. 

The first test which stops provides some C G ^: put aj ~ C . 

Let x be a random point whose iterates are covered by ^, and s^q, . . . , s,;„_j be 
ideal points which e-shadow the first n iterates of x. We claim that A will halt on 
(e, io, ■ • ■ , hi-i)- Indeed, as T^x belongs to some C G ^, Cr\B{si. , e) is a non-empty 
open set and then contains at least one ideal point s, which will be eventually dealt 
with. 

We now compare the symbolic orbit of x with the symbolic sequence computed by 
A. A discrepancy at rank j can appear only if T^x G (90^*^ ■ Indeed, if T^x ^ (9^)^^ 
then B{T^x,2e) C C where C is the cell T^x belongs to. As d{s,^,T^x) < e, 
B{si^ , e) C B{x, 2e) C C, so the algorithm gives the right cell. 

Now, as X is typical, 

limsup < n : T^x G (9?)^'} < fi (WF) < " 

n — yoo ri V / 

so there is some riQ such that for all n > uq, < n : T^x G (9^)^^} < a. 

This implies that for all n > uq and ideal points s^^, . . . , s^^ ^ which e-shadow the 
first n iterates of x and with minimal complexity, the algorithm A{e, iq, ■ ■ ■ , in-i) 
produces a symbolic string u which differs from the symbolic orbit w of a; of length 
n with a density of errors < a. As K{u) < K[e) + lCn{x,T,t) and af{l/a) < /?, 
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applying lemma 5.0.1 gives: 

-K{Ux)) = -K{v) < -K{u) + af{l/a) + - 
n n n n 

< - (/C„ (x, T, e) + A-(e) + c') + /? + - 
n n 

where c' is independent of n. Taking the limsup as rt — > cxd gives: 

/C^(x,T|0 <^a:,T,e)+/3 

□ 

Combining theorems 5.0.1 and 4.4.2, we obtain a version of Brudno's theorem 
(theorem 1) for Martin-L6f random points. 

Corollary 5.0.1. Let T be an ergodic endomorphism of the computable probability 
space {X,fi), where X is compact. Then for every Martin-Lof random point x: 

lC{x,T)^h^{T) 

6. Topological entropies 

Bowen's definition of topological entropy is reminiscent of the capacity (or box 
counting dimension) of a totally bounded subset of a metric space. In order to 
find relations with orbit complexity we will also use another characterization of 
topological entropy, expressing it as a kind of Hausdorff dimension. We first present 
Bowen's definition. 

In this section, X is a metric space and T : X ^ X & continuous map. 

6.1. Entropy as a capacity. We recall the definition: for n > 0, let us define the 
distance dn{x,y) = niax{d{T^x,T^y) : < i < n} and the Bowen ball Bn{x,e) = 
{y : dn{x,y) < e}, which is open by continuity of T. Given a totally boimded set 
Y C X and numbers n > 0, e > 0, let N{Y, n, e) be the minimal cardinality of a 
cover of Y by Bowen balls Bn{x, e). A set of points E such that {Bn{x, e) : x G E} 
is a cover of Y is also called an (n, e)-spanning set of Y. One then defines: 

h (rrv \ V log N{Y,n,e) 
III (1 ,Y,e)— hm sup 

n — >oo 1^ 

which is non-decreasing as e — > 0, so the following limit exists: 

hi{T,Y) = lim/ii(r,r,e). 

When X is compact, the topological entropy of T is h{T) = hi{T, X). It measures 
the exponential growth-rate of the number of distinguishable orbits of the system. 

Remark 6.1.1. The topological entropy can be defined using separated sets in- 
stead of open covers: a subset A of X is (n, e)-scparatcd if for any distinct points 
x,y G A, dn{x,y) > e. Let us define M{Y,n,e) as the maximal cardinality of 
an (n, e)-separated subset of Y. It is easy to see that AI{Y,n,2e) < N(Y,n,e) < 
M{Y, n, e), and hence hi{T, Y) can be alternatively defined using M{Y, n, e) in place 
of iV(y,7i,e). 
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6.2. Entropy as a dimension. It is possible to define a topological entropy which 
is an analog of Hausdorff dimension. His definition coincides with the classical one 
in the compact case. Hausdorff dimension has stronger stability properties than 
box dimension, which has important consequences, as we will see in what follows. 
We refer the reader to [Pes98], [HK02] for more details. 

Let X be a metric space and T : X X a continuous map. The e-size oi E C X 
is 2~* where 

s = sup{n > : diam^TE) < e for < i < n}. 
It measures how long the orbits starting from E are e-close. As e decreases, the 
e-size of E is non-decreasing. The 2e-size of a Bowen ball Bn{x, e) is less than 2~". 

In a way that is reminiscent from the definition of Hausdorff measure, let us 
define 

mKF,6)=inf|5](e-size(t/)r| 
[ueg ) 

where the infimum is taken over all countable covers Q oi Y hy open sets of e- 
size < S. This quantity is monotonically increasing as d tends to 0, so the limit 
m'^{Y,e) := lim5^Q+ mg{Y,e) exists and is a supremum. There is a critical value 
So such that m'^{Y,e) = oo for s < sq and m^{Y,e) = for s > sq. Let us define 
h2{T, Y, e) as this critical value: 

h2{T, Y, e) := inf {s : m%Y, e) = 0} = sup {s : m''{Y, e) = oo} . 

As less and less covers are allowed when e ^ (the e-size of sets does not decrease) , 
the following limit exists 

h2{T,Y) := lim h2{T,Y,e) 

e— »0+ 

and is a supremum. In [Pcs98], it is proved that: 

Theorem 6.2.1. When Y is a T-invariant compact set, hi{T,Y) ~ h2{T,Y). 
In particular, if the space X is compact, then h{T) — hi{T,X) ^ h2(T,X). 

6.3. Orbit complexity vs entropy. Now we prove the main theorem of the sec- 
tion: 

Theorem 6.3.1 (Topological entropy vs orbit complexity). Let X be a compact 
computable metric space, and T . X ^ X a computable map. Then 

h{T) = sup £(.T, T) = sup IC{x, T). 

xex xex 

In order to prove this theorem, we define an effective version of the topological 
entropy, which is strongly related to the complexity of orbits. 

6.3.1. Effective entropy as an effective dimension. Before defining an effective ver- 
sion, we give a simple characterization which will accommodate to effectivisation. 

Definition 6.3.1. A null s-cover of y C X is a set i5 C such that: 

(1) E(.,„,p)eij2-^"<^. 

(2) for each k,p £ N, the set {_B„(si, 2~p) : (i, n,p) G E,n > k} is a cover of Y. 

The idea is simple: every null s-cover induces open covers of arbitrary small size 
and arbitrary small weight. Remark that any null s-cover of Y is also a null s'-cover 
for all s' > s. 
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Lemma 6.3.1. h2{T,Y) = inf{s : Y has a null s-cover}. 

Proof. Suppose s > h2{T,Y). We fix p, fc e N and put e = 2-p and S = 2"'=. 
As mg{Y,e) = 0, there is a cover {Uj^k,p)j of Y by open sets of e-size 5j^k,p < S 
with J2j^jkp < 2"^*^+''''. Let Si be any ideal point in Uj^k.p- If Sj,k,p > 0, then 
^j,k,p = 2~" for some n. If 6j^k.p ~ 0, take any n > {j + k + p)/s. In both 
cases, Uj_k,p is included in the Bowen ball Bn{si,e). We define Ek^p as the set 
of {i,n,p) obtained this way, and E = Ufcp^fe.p- construction, for each k,p, 
{S„(s,,2-P) : {i,n,p) eE,n>k} is a cover of F. Moreover, E(z.„,p)e£;, ^ 2^^" < 

Ej Slk,p + Ej 2-(^+^-+P) < 2-('=+P)+2, so Ei.,n,p)eE 2-^" < oo. ■ ' 

Conversely, if Y has a null s-covcr E, take e,S > Q and p. A: such that e > 2^^^^^ 
and 5 > 2~'^. For all k' > k, the family {Bn{si, 2"^) : {i, n,p) & E,n> k'} is a cover 
of Y by open sets of e-size smaller than 2^" < S. Moreover, J2{i n p}eE n>k' 2^^*" 
tends to as k' grows, so m|(y, e) = 0. It follows that s > h2{T, Y). □ 

By an effective null s-cover, we mean a null s-cover E which is a r.e. subset of 

Definition 6.3.2. The effective topological entropy of T on 1" is defined by 

h'^{T, Y) = inf {s : Y has an effective null s-cover} 

As less null s-covers are allowed in the effective version, h2{T,Y) < h'^{T^Y). 
Of course, iiY <ZY' then hf{T, Y) < hf{T, Y'). We now prove: 

Theorem 6.3.2 (Effective topological entropy vs lower orbit complexity). Let X 
he an effective metric space and T : X X a continuous map. For all Y <Z X , 

hf{T,Y) = sup£(x,T) 

which implies in particular that /i2^(T, {x}) = £(x,T): the restriction of the 
system to a single orbit may have positive effective topological entropy. 

This kind of result has already been obtained for the Hausdorff dimension of 
subsets of the Cantor space, proving that the effective dimension of a set A is the 
supremum of the lower growth-rate of Kolmogorov complexity of sequences in A 
(which corresponds to theorem 6.3.2 for sub-shifts). This remarkable property is a 
counterpart of the countable stability property of Hausdorff dimension (dim Y — 
supidimFi when |J, >i = Y) (see [CH94], [MayOl], [Lut03], [Rei04], [Sta05]). 

Theorem 6.3.2 is a direct consequence of the two following lemmas. 

Lemma 6.3.2. Let a > and Y^ {x : ICfx, T)<a]. One has hf{T, Y^) < a. 

Proof. Let /3 > a be a rational number. We define the r.e. set E — {{i,n,p) : 
K{i,n,p) < Pn}. Let p S N and e = 2"^. li x G Y^ then £(x,T,e) < a < f3 so 
for infinitely many n, there is some Si such that x e Bn{si,e) and I{(i,n,p) < 
/3n. So for all k, {Bn{si,2~P) : {i,n,p) e E,n > k} covers Yq,. Moreover, 

E is then an effective null /3-cover of Ya, so h'^{T^ Ya) < /3. And this is true for 
every rational (3 > a. □ 

Lemma 6.3.3. Let Y C X . For all x G Y, IC{x, T) < hf{T, Y). 
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Proof. Let s > hf{T,Y): Y has an effective null s-cover E. As E(»,„.p)6is 2""" < 
oo, by the coding theorem K(i, n,p) < sn + c for some constant x, which docs not 
depend on i, n,p. If x E Y, then for each p, k, x is in a ball Bn{si, 2~p) for some 
n > k with {i,n,p) G E. Then lCn{x,T,2~P) < sn + c for infinitely many n, so 
IC{x,T,2^P) < s. As this is true for all p, lC{x,T) < s. As this is true for all 
s > ft-2^(T, y), we can conclude. □ 

Proof of theorem 6.3.2. By lemma 6.3.3, a := sup^gy /C(x, T) < hf{T,Y). Now, 
as y C Yc, hf{T, Y) < hf{T, Y^) < a by lemma 6.3.2. □ 

The definition of an effective null a-cover involves a summable computable se- 
quence. The universality of the sequence 2"^^*^ among summable lower semi- 
computable sequences is at the core of the proof of the preceding theorem, which 
states that there is a universal effective null a-cover, for every a > 0. In other 
words, there is a maximal set of effective topological entropy < a, and this set is 
Yc, = {x ex : !C{x,T) < a}. 

The definition of the topological entropy as a capacity could be also made effec- 
tive, restricting to effective covers. Classical capacity does not share with Hausdorff 
dimension the countable stability. For the same reason, its effective version is not 
related with the orbit complexity as strongly as the effective topological entropy 
is. Nevertheless, a weaker relation holds, which is sufficient for our purpose: the 
upper complexity of orbits is bounded by the effective capacity. We do not develop 
this and only state the needed property (which implicitly uses the fact that the 
effective capacity coincides with the classical capacity for a compact computable 
metric space): 

Lemma 6.3.4. Let X be a compact computable metric space. For all x G X, 
lC{x,T) < hi{T,X). 

Proof. We first construct a r.e. set i? C N"^ such that for each n, p, {si : {i,n,p) £ 
E} is a (n, 2^^')-spanning set and a (n, 2~P~^)-separated set. Let us fix n and p 
and enumerate En^p = {i : {i,n,p) G E}, in a uniform way. The algorithm starts 
with 5' = and i = 0. At step i it analyzes st and decides to add it to S or not, 
and goes to step i + En,p is the set of points which are eventually added to S. 
Step i: for each ideal point s G S, test in parallel dn{si,s) < 2~p~^ and 
dn{si,s) > 2~P~'^: at least one of them must stop. If the first one stops 
first, reject Si and go to Step i -|- 1. If the second one stops first, go on with 
the other points s G S: if all S has been considered, then add Si to S and 
go to Step i -I- 1. 

By construction, the set of selected ideal points forms a {n, 2^^'^^)-separated set. 
If there is x G AT which is at distance at least 2^p from every selected point, then let 
Si be an ideal point Si with dn{x, Si) < 2~p~^: Si is at distance at least 2~P'^ from 
every selected point, so at step i it must have been selected, as the first test could 
not stop. This is a contradiction: the selected points form a (71, 2^P)-spanning set. 

From the properties of £'„,p it follows that N{X, n, 2"^) < \E„^p\ < M{X, n, 2-p-'^), 
and then 

sup limsup — log \En ml ) = hi(T, X) 
p \ n 'J 

If /3 > hi{T,X) is a rational number, then for each p, there is fc G N such that 
log \En^p\ < (in for all n > k. 
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Now, for Si e En,p, K{i) < log|i?„,p| + 2 loglog |i?„^p| + K{n,p) by proposi- 
tion 2.7.1. Take x € X: a; is in some Bn{si,2~P) for each n, so IC{x,T,2^p) < 
limsup„ i log \En,p\ < /3 as log \E„.p\ < (3n for all n > k. As this is true for all p 
and all (3 > hi{T, X), lC{x, T) < hi{T, X) and this for all xeX. □ 

We are now able to prove theorem 6.3.1. Combining the several results estab- 
lished above: 

h,{T,X)^h2{T,X) < hf{T,X) = snp,^xU^.T) < sup^^j,lC{x,T) < h,{T, X) 
(theorem 6.2.1) (theorem 6.3.2) (lemma 6.3.4) 

and the statement is proved. 
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